Skip to content

knowledge: add GPU measurement-validity and profiler attribution guidance - #456

Draft
BeatSeat wants to merge 3 commits into
InferMatrixLab:mainfrom
BeatSeat:knowledge/measurement-validity
Draft

BeatSeat wants to merge 3 commits into
InferMatrixLab:mainfrom
BeatSeat:knowledge/measurement-validity

Conversation

@BeatSeat

@BeatSeat BeatSeat commented Nov 15, 2026 •

Copy link
Copy Markdown

Governing Issue and delivery slice

Refs #455

  • Accepted scope / delegation: task: add GPU measurement-validity and profiler-attribution guidance to inference-benchmark-design #455 was accepted with conditions by the steward on behalf of Dong1017 (goal-acceptance owner). This PR addresses those conditions; no closure is claimed.
  • What this PR delivers: A 175-line GPU measurement-validity reference, a six-line skill pointer, pinned public-source metadata, matrix navigation in dedicated sections, three regenerated projections and the installed-resource fixture update to 929 (see below).
  • What it does not deliver: A benchmark implementation, new skill, campaign threshold, empirical validation or performance claim.

GPU E2E timings can move with clock, thermal, power and sharing conditions; profiler replay can introduce different execution conditions. The reference gives a procedure for retaining paired timings, checking conditional within-run attribution and Amdahl predictions, and connecting gains to work counters and a predicted profiler signature. It covers timing/cache/baseline/JIT pitfalls and keeps the four supplied numerical observations explicitly unverified.

The existing inference-benchmark-design owner already defines comparable workloads, measurement integrity and evidence interpretation. Under the knowledge-base ownership contract, this is an owner-local reference extending that procedure; it does not need a separate skill identity or registry entry. Candidate state, empirical not_run and efficacy not_evaluated remain explicit.

Changes since review

Rebased onto main 6c7e2379 (after #453, #458, #459, #342, #462, #463, #461, #465, #435, #382 and two direct commits) and regenerated from the new source commit.

Knowledge authorship declaration

{
  "schema_version": 1,
  "head_sha": "4458a0530a0e2a312eb6ce98af190afe5091b97e",
  "human_only": false,
  "material_ai_authors": [
    {
      "tool": "Codex CLI",
      "model_id": "gpt-6.1-sol",
      "effort": "xhigh",
      "role": "author"
    }
  ]
}

Changed artifacts and evidence

Requirement Actual change Check / evidence and result Remaining gap
Concise, source-grounded procedure methods/inference-benchmark-design/references/gpu-measurement-validity.md (175 lines); SKILL.md adds six lines Ten public-source identities; citation labels resolve; retained document hashes checked against retrieved bytes No GPU/profiler or efficacy execution
Attribution and pitfalls Same-run region shares, conditional internal clock, serial/critical-path limits, Amdahl check, count metrics, explanation-first adoption, five timing-trap categories Official versioned NVIDIA/PyTorch docs, pinned Triton source and robust-kbench v1 inspected Normalization/adoption is an unverified author proposal; legacy counters require target queries
Preserve evidence state Owner metadata and four task-supplied observations Normalized checkout/package reads succeed; observations labelled unverified with unavailable raw evidence No supplied campaign artifacts
Keep projections and packaging consistent Dedicated GPU measurement-validity sections in both matrices; target generated record, _provenance.json, coverage view; tests/check_installed.py count 929 Both generators and checks pass; installed-wheel inventory and new-reference byte checks pass Independent review pending

Canonical skill paths above are relative to src/inferrsi/knowledge/data/skill_library/. The registry, unrelated skill records and generated navigation README remain unchanged.

Checks and dependencies

All Python commands used the requester-specified interpreter and checkout PYTHONPATH. Final Git-dependent checks used a temporary Git 2.49.1 build, with the repository tools unchanged and native --no-lazy-fetch support.

  • tools/build_skill_index.py --source-revision ee8c924f61f0a8f17338b8b25909a410398067ad, then --check: passed (exit 0), 80 records current; generated churn limited to the target record, _provenance.json and the coverage view.
  • tools/build_skill_coverage.py, then --check: passed (exit 0). The coverage view now labels the source under GPU measurement validity and profiler attribution, not issue [RFC] Benchmark selection by task for InferRSI #60.
  • python -m pytest tests/test_skill_catalog.py tests/test_skill_index.py tests/test_skill_library.py tests/test_skill_library_readers.py tests/test_skill_coverage_view.py tests/test_knowledge_*.py --basetemp <temporary-test-dir>: 485 passed, 5 skipped, 112 subtests passed at the final head.
  • python -m ruff check src tests: passed.
  • Built a wheel from a clean git archive of the final head (pip build isolation; SETUPTOOLS_SCM_PRETEND_VERSION=0.0.0, because the evidence-minicpm-seedtts-dec7a734-v1 tag on main breaks setuptools-scm version parsing), installed it noneditably to a temporary target, and ran tests/check_installed.py outside the checkout with isolated imports: passed, 929 resource files.
  • tools/knowledge_declaration_paths.py --base origin/main --head HEAD: passed (exit 0).
  • Offline assess() from tools/check_knowledge_model_policy.py against this PR body, final head, base-relative paths and the trusted main policy: compliant, declaration compliance only (7 protected paths).
  • git diff --check, clean-worktree check, DCO sign-off and Agent-assisted-by trailer inspection on both commits: passed.
  • Generated records were produced only by the repository generators, with no manual edits. Claude Code (claude-opus-5-5) executed the generator commands as an operator, as the regeneration commit trailer records; under the policy, running a generator is not material authorship, so the declaration lists the source author only.

Resolved validation failures: System Git 2.43 lacks --no-lazy-fetch; generation and Git-dependent tests used Git 2.49.1. A first wheel built from the working tree picked up a stale local build/ directory containing retired root modules; the wheel was rebuilt from a clean git archive and the installed check passed.

Skipped / not run: One browser test requires unavailable Playwright; two fresh-venv tests remain opt-in; two NTFS-junction cases are Windows-only. The separate noneditable wheel check above ran successfully. Full ready/nightly CI, independent review, GPU profiling, model quality, L1–L4 campaign execution and efficacy evaluation were not run.

Limits

  • The attribution/adoption procedure is proposed guidance. Its internal-clock correction requires common drift and an unaffected reference; overlapping kernels require critical-path analysis. No numeric gate is introduced.
  • The four numerical cases are task-supplied author reports with no raw samples/configurations/profiles supplied. They are neither public-source findings nor evidence of efficacy.
  • Metric spellings have an official legacy NCU mapping; availability and semantics still need inspection on the actual GPU/tool version. PyTorch's naming advice is explicitly an inference from its documented module/build identity contract, not a claim that every duplicate-name load fails.
  • The unversioned SMI page is identified by its inspected HTML hash in metadata. Raw third-party documents/code are not redistributed. Source review does not grant execution permission or promote this skill.
  • The wheel build emitted setuptools package-discovery warnings for data directories; the complete installed-resource inventory check nevertheless passed. Packaging configuration was not expanded in this slice.
  • Follows the two-step workflow in docs/knowledge-contribution-workflow.md: canonical files are committed first (ee8c924f, the pinned source_revision), then records are regenerated from that commit and committed (f89f7667). A third commit (4458a053) only updates the installed-resource count in tests/check_installed.py, which is not a canonical or protected path. The source revision is an ancestor of the head, so no separate bundle is needed.

Remaining work and Issue closure

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Skill evaluation

PLANNED: not executed; there are no task outcomes yet.
No approval, model efficacy or current-head test result is implied.

  • Next: run the pinned source/index --check below at head 8c0b894a2dc9b8572700ca3f80b119e75f2377db.
  • Blockers: Executable snapshot entries are not supported. No unsafe fallback; details below.
  • Selected comparisons: 0 · Estimated task calls: 0. Uncovered/unsupported changes: 60; Canonical library paths: 3; Generated projection paths: 2; Other paths: 55. These are paths, not independent claims.
  • Return: command, input/source pin, exit status, outputs/failures and untested claims for independent verified/refuted/unverified disposition.
Author verification steps (nonpaid; not yet run)
  1. Bind the checkout. Run git rev-parse HEAD; it must equal 8c0b894a2dc9b8572700ca3f80b119e75f2377db. If it differs, obtain a current plan; do not reuse old results.
  2. Check source/index consistency. At that PR checkout, using the reviewed repository tool and its documented development dependencies, run:
python tools/build_skill_index.py --source-revision 3671a98ad340fb19e6595d1c8d818c7e9ca2577a --check

The declared source pin matches this head's canonical library bytes. The command has not run: retain its exit code and full drift/success output. It checks the generated projection, not claim truth or skill efficacy.
No supported automatic comparison is selected. Review the canonical changes and explain which claims need evidence, are unaffected, or remain unverified; do not treat generated records as separate behavioral claims.

  • inference-benchmark-design
    Selected by the trusted path mapping: src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md; src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json (+1 more in the plan).
    Unsupported: Executable snapshot entries are not supported. No snapshot prepared; task metadata is in details. Do not relax snapshot safety.
    Uncovered scope by location: Canonical library paths: 3; Generated projection paths: 2; Other paths: 55. Path counts are not claim counts; no metadata-only conclusion is inferred.
  1. Return evidence for independent review: head, check/task/source pin, command, environment, exit status, outputs/failures and untested claims. Label scripted controls. Request each claim's disposition: verified, refuted or unverified; this plan proves neither model provenance nor benefit.
    Check recipes and evidence limits

Evidence and logs · Retention: 30 days.
Retained artifacts are verifiable only before Unix time 1797349159 while the linked artifact is available. Expired evidence is unverified.

Plan bindings, budget, frozen exam and advisory details

Advisory: obtain missing evidence

Retained evidence availability: planned / unexecuted. No approval or merge authorization is issued.
Applicable only while these revisions and inputs match the current PR.
Paid execution requires trusted label authorization and cumulative PR reservation. Unknown accounting blocks new paid calls; planning is not execution.
Pipeline evidence only; offline fixed outputs do not establish skill efficacy.
Unchanged-correct is valid neutral evidence, not incremental benefit.
Content/source review and authorship admission remain separate.
Trusted regrading checks retained answers, not model/hardware provenance or a hostile executor; this is not an efficacy certification.

  • Retained evidence is planned / unexecuted.
  • No supported comparison cases were selected.
  • Uncovered/unsupported scope needs a case or explicit reviewer disposition.
    Plan SHA-256: 497cd20678afbe369cdea4b5bed22ff96317693ed9ebbd547f1c396d7c721d50
    Base: cd8fcb452bd2c7969a7a6e2544aa1cb5f8ad5332 · Head: 8c0b894a2dc9b8572700ca3f80b119e75f2377db
    Trusted implementation: cd8fcb452bd2c7969a7a6e2544aa1cb5f8ad5332

Model/settings: {"effort":"not_applicable","name":"claude-opus-5-5"}
Budget: {"comparisons":0,"estimated_calls":0,"limits_per_comparison":{"campaign_seconds":600,"max_task_calls":2,"repeats":1,"timeout_seconds":180,"token_stop_new_trials":200000},"provider_config_sha256":"b28d80f8194fbe0f5a9f01515efb24b218979fd45f702776560da276f4b30d04","reservation":null}

Skill Comparison Cases Estimated calls
inference-benchmark-design unsupported snapshot: Executable snapshot entries are not supported — 0

Selection/task metadata (not a relevance verdict):

{"adapter":"audit","basis":"trusted_path_mapping","matched_paths":["src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md","src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json","src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md"],"tasks":[{"evidence_scope":"http_fixed_workload","id":"audit-threshold","request":"Audit the fixed input report using its contract. Compute the median of per-trial request means, including every request in each trial, and the percentage reduction relative to the baseline aggregate.","split":"holdout"}]}

Automatic execution / cumulative PR budget:

{"reason":"Trusted configuration or pricing missing, invalid or expired","remaining_usd":"unknown","reserved_or_unresolved_usd":"unknown","spent_or_legacy_usd":"unknown","state":"blocked"}

Projection check selection (not an executed result):

{"declared_in":"src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/_provenance.json","source_revision":"3671a98ad340fb19e6595d1c8d818c7e9ca2577a","status":"ready"}

Provider/settings: {"client":{"api_version":"2023-06-01","endpoint":"https://yibuapi.com/v1/messages","kind":"anthropic-messages-full-context-v1","max_output_tokens":4096,"max_read_bytes":32768,"max_request_bytes":262144,"max_requests":1,"max_tool_calls":12,"provider":"yibuapi"},"limits":{"token_stop_new_trials":200000},"model":{"effort":"not_applicable","name":"claude-opus-5-5"},"pricing":{"currency":"USD","expires_at":1793271228,"group":"default","max_input_tokens":1000000,"model":"claude-opus-5-5","rate_evidence":"https://github.com/InferMatrixLab/InferRSI/issues/215#issuecomment-6467531341","rates":{"cache_read":"0.4","cache_write":"10","input":"8","output":"40"},"request_fee":"0","unit":"per_million_tokens"},"spend_limit":{"amount":"200","currency":"USD"}}

Uncovered / unsupported paths (60)
  • uncovered: "AGENTS.md"
  • uncovered: "CHANGELOG.md"
  • uncovered: "docs/contracts/current-design.md"
  • uncovered: "docs/module-layout.md"
  • uncovered: "docs/reference-matrix.md"
  • uncovered: "docs/searchspace-matrix.md"
  • uncovered: "docs/skill-coverage.md"
  • uncovered: "src/inferrsi/_json_v1.py"
  • uncovered: "src/inferrsi/apps/api.py"
  • uncovered: "src/inferrsi/apps/archive.py"
  • uncovered: "src/inferrsi/apps/artifact_reader.py"
  • uncovered: "src/inferrsi/apps/bridge.py"
  • uncovered: "src/inferrsi/apps/context_provider.py"
  • uncovered: "src/inferrsi/apps/legacy/evaluation.py"
  • uncovered: "src/inferrsi/apps/legacy/run.py"
  • uncovered: "src/inferrsi/apps/result.py"
  • uncovered: "src/inferrsi/apps/views/attempt.py"
  • uncovered: "src/inferrsi/arguments.py"
  • uncovered: "src/inferrsi/claim.py"
  • uncovered: "src/inferrsi/contracts.py"
  • uncovered: "src/inferrsi/contracts/init.py"
  • uncovered: "src/inferrsi/contracts/claim.py"
  • uncovered: "src/inferrsi/contracts/json_v1.py"
  • uncovered: "src/inferrsi/contracts/runspec.py"
  • uncovered: "src/inferrsi/execution.py"
  • uncovered: "src/inferrsi/harness/init.py"
  • uncovered: "src/inferrsi/harness/execution_service.py"
  • uncovered: "src/inferrsi/harness/execution_spec.py"
  • uncovered: "src/inferrsi/harness/run_store.py"
  • uncovered: "src/inferrsi/harness/runner.py"
  • uncovered: "src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/_provenance.json"
  • uncovered: "src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/inference-benchmark-design.json"
  • unsupported: "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md"
  • unsupported: "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json"
  • unsupported: "src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md"
  • uncovered: "src/inferrsi/runspec.py"
  • uncovered: "tests/_gate_policy_support.py"
  • uncovered: "tests/architecture_real_ports.py"
  • uncovered: "tests/check_installed.py"
  • uncovered: "tests/h01_run_store_child.py"
  • uncovered: "tests/test_apps_assess.py"
  • uncovered: "tests/test_apps_evidence.py"
  • uncovered: "tests/test_apps_execution_binding.py"
  • uncovered: "tests/test_apps_output.py"
  • uncovered: "tests/test_apps_prepare.py"
  • uncovered: "tests/test_claim.py"
  • uncovered: "tests/test_contracts_package.py"
  • uncovered: "tests/test_dashboard_workflow.py"
  • uncovered: "tests/test_first_wave_isolation_policy.py"
  • uncovered: "tests/test_flare_adapter.py"
  • uncovered: "tests/test_harness_execute.py"
  • uncovered: "tests/test_harness_run_store.py"
  • uncovered: "tests/test_knowledge_model_policy.py"
  • uncovered: "tests/test_legacy_run_split.py"
  • uncovered: "tests/test_module_boundaries.py"
  • uncovered: "tests/test_package_root_inventory.py"
  • uncovered: "tests/test_smoke.py"
  • uncovered: "tests/test_target_module_import_gate.py"
  • uncovered: "tests/test_trial_check_reports.py"
  • uncovered: "tests/test_trial_checks.py"

Copy link
Copy Markdown
Collaborator

Thanks for the careful caveats and the two-step generation. I reviewed this against current main cd8fcb452bd2c7969a7a6e2544aa1cb5f8ad5332 and head 8c0b894a2dc9b8572700ca3f80b119e75f2377db (branch point a92c77ccaed68426c83154cff7619745f34d0058 / source revision 3671a98ad340fb19e6595d1c8d818c7e9ca2577a, then the regeneration commit).

  1. Matrix section. The new row in docs/reference-matrix.md was added under "Task-specific benchmark selection (issue [RFC] Benchmark selection by task for InferRSI #60)", so docs/skill-coverage.md also labels it as an [RFC] Benchmark selection by task for InferRSI #60 source, and the searchspace-matrix paragraph was inserted under the same heading. This reference is general GPU measurement methodology, not a task-selection source for [RFC] Benchmark selection by task for InferRSI #60. Please move the matrix row and the searchspace note to a dedicated short section so the generated coverage label is accurate.

  2. Rebase onto current main. The branch is based on a92c77ccaed68426c83154cff7619745f34d0058. Package shared technical contracts and retire root modules #453 (cd8fcb452bd2c7969a7a6e2544aa1cb5f8ad5332) landed afterwards and added C1 assertions to tests/check_installed.py while keeping the installed-resource count at 923. A merge may apply the 923 → 924 change cleanly, but please rebase and, if the source-revision commit changes, regenerate the records and update the declaration head_sha before merge.

I also checked: the new reference is 175 lines; its SHA-256 78d750321e65b69f1cc2006997d2e66494284e65cc7eb55d9f2c88e17aedb87c is present in the generated skill record; the Triton v3.2.0 tag is 9641643da6c52000c807b5eeed05edaec4402a67; and do_bench matches the write-based flush / preliminary-estimate / per-sample event description. Status remains candidate / empirical not_run. I did not re-download the NVIDIA/PyTorch documents to re-check the retained hashes, and I did not rerun the generators or the installed-wheel check. #455 is still Proposed, with no acceptance owner yet.

I am not approving this head until the matrix placement is fixed.

@Dong1017 Dong1017 added area:knowledge Knowledge line: packs, skill library, knowledge admission kind:task Bounded delivery task with a delegated slice labels Nov 15, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR review verdict — DO NOT PASS
Reviewed base cd8fcb452bd2c7969a7a6e2544aa1cb5f8ad5332 and head 8c0b894a2dc9b8572700ca3f80b119e75f2377db (draft).

Acting reviewer: cursor[bot] via standing InferRSI steward delegation. Binding formal review to current head.

Blockers before Approve / ready-for-review:

  1. Matrix placement. The new docs/reference-matrix.md row for GPU measurement validity sits under ## Task-specific benchmark selection (issue #60) (around L444–L451). The matching searchspace note is under the issue #60 heading as well. This is general GPU measurement methodology, not an #60 task-selection source. Move the matrix row and searchspace paragraph to a dedicated short section (same pattern as ## Faster CosyVoice evaluation source context), then regenerate coverage/records so docs/skill-coverage.md does not label it as an #60 source.
  2. Rebase onto current main. Head diverges: ahead 2 / behind 1 vs cd8fcb45 (#453). Rebase, and if the source-revision commit changes, regenerate projections and update the declaration head_sha.
  3. Governing issue still Proposed. #455 has no recorded owner acceptance yet. Steward decision will be posted on #455; keep this draft until acceptance + the matrix/rebase fixes land.

Checked / not blocking alone: docs-only slice; candidate / empirical not_run preserved; Triton pin and do_bench flush description match prior notes; installed-count 923→924 is coherent with a single new reference. I did not re-download NVIDIA/PyTorch docs or re-run the installed-wheel check.

Re-request review after the blockers are addressed on a new head.

Copy link
Copy Markdown
Collaborator

I re-checked after the steward decision on #455: the documentation-only slice is now Accepted (with conditions), and the goal-acceptance owner is Dong1017. That external blocker is cleared.

This note is for the same head 8c0b894a2dc9b8572700ca3f80b119e75f2377db (no new commits since my previous comment). The remaining blockers are unchanged:

  1. Matrix placement. The new docs/reference-matrix.md row and the searchspace note are still under the issue [RFC] Benchmark selection by task for InferRSI #60 heading, so the generated coverage label would still be wrong. Please move them to a dedicated short section and regenerate records.
  2. Rebase onto current main. The branch point is still a92c77ccaed68426c83154cff7619745f34d0058. Since then, main has moved from Package shared technical contracts and retire root modules #453 (cd8fcb452bd2c7969a7a6e2544aa1cb5f8ad5332) to refactor(C2): relocate resource facade and split evidence layout #458 (5db1bc94042f386f0b4b35c3e6fe91598fef118f). Please rebase onto this main, and if the source-revision commit changes, regenerate records and update the declaration head_sha.

The “require an explanation before adoption” wording should stay clearly candidate guidance, as accepted on #455. I am not approving this head. cursor[bot] also left CHANGES_REQUESTED / DO NOT PASS on the same head for these items. Please request a re-review once they are addressed (likely on a new head).

@BeatSeat

Copy link
Copy Markdown
Author

Addressed on the new head 6ec20789 (based on current main 016c7c7d):

  1. Matrix placement. The docs/reference-matrix.md row and the docs/searchspace-matrix.md note now sit in dedicated "GPU measurement validity and profiler attribution" sections, outside the issue [RFC] Benchmark selection by task for InferRSI #60 section. The regenerated docs/skill-coverage.md labels the source under that section.
  2. Rebase. Rebased onto 016c7c7d. Conflicts with Extend VLA observation-to-action benchmark guidance #342 (VLA guidance) in metadata.json, reference-matrix.md and tests/check_installed.py were resolved by keeping both additions; the installed-resource count is 928. The source commit changed, so records were regenerated with the two-commit flow (source 3a6dc01c, regeneration 6ec20789) and the declaration head_sha was updated.
  3. Candidate wording. "Require an explanation before adoption" remains author-proposed candidate guidance, not a repository-wide gate; the four task-supplied figures remain unverified.

build_skill_index.py --check, build_skill_coverage.py --check, the knowledge test set and an installed-wheel check_installed.py pass locally; details are in the PR body. Please re-review on the new head.

amy-why-3459
amy-why-3459 previously approved these changes Nov 16, 2026

@amy-why-3459 amy-why-3459 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base 016c7c7df023944f0f46d8e777d6d31f1df1764c and head 6ec207892ed4204b9e14d1fc6166681e048b301e (draft).

The previous blockers on 8c0b894a2dc9b8572700ca3f80b119e75f2377db are resolved:

  • The docs/reference-matrix.md row and the docs/searchspace-matrix.md note now sit in a dedicated "GPU measurement validity and profiler attribution" section, and docs/skill-coverage.md labels the source under that section rather than issue #60.
  • The branch is rebased onto current main 016c7c7d (ahead 2 / behind 0). Provenance source_revision is 3a6dc01c1139d1d97fad52b99cc39be61cad520d, the regeneration commit is this head, and the installed-resource count is 928 (927 on main plus the new reference).
  • #455 remains accepted with conditions (steward comment 6897438871). Candidate / empirical not_run wording is preserved; the explanation-first adoption language stays author-proposed guidance.

declarations and knowledge-model-policy passed on this head. I did not re-download the NVIDIA/PyTorch documents to re-check retained hashes, and I did not rerun the generators or the installed-wheel check.

cursor[bot]
cursor Bot previously approved these changes Nov 16, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base 016c7c7 and head 6ec2078 (draft).

Re-review after author push from prior CHANGES_REQUESTED on 8c0b894. Prior blockers cleared on this head:

  1. Matrix row and searchspace note now live under dedicated "GPU measurement validity and profiler attribution" sections (not issue #60); skill-coverage labels match.
  2. Branch was rebased onto 016c7c7 with regenerated records / updated declaration head_sha.
  3. Governing #455 acceptance recorded earlier.

Scope: documentation/knowledge slice only; candidate/empirical not_run preserved. Declarations check green.

Merge process (not content blockers on this draft head): still draft; head is behind current main again after later merges — rebase before marking ready; run ready-lane CI on the rebased tip. skill-eval plan failure on this head is setuptools_scm tag-parse infra (evidence-minicpm-… tag), not a content defect in this slice.

Acting steward re-review while hosted Dong-bot is on usage_limit; formal Approve as cursor[bot].

BeatSeat and others added 3 commits November 16, 2026 21:27
…ance

Extend inference-benchmark-design with an owner-local reference on clock and
cache controls, within-run attribution, work-count signatures and timing
traps. Candidate guidance; empirical validation not run.

Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=xhigh] (author)
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>
Records generated by tools/build_skill_index.py and tools/build_skill_coverage.py
from source revision ee8c924.

Agent-assisted-by: Claude Code / claude-opus-5-5 (ran generators)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>
…s reference

main 6c7e237 (InferMatrixLab#382) added one packaged knowledge resource without updating
the installed-resource assertion, so main installs 928 files against an
expected 927. With the measurement-validity reference the installed count is
929, verified on a wheel built from a clean archive of this head.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>
@BeatSeat
BeatSeat dismissed stale reviews from cursor[bot] and amy-why-3459 via 4458a05 November 16, 2026 13:35
@BeatSeat
BeatSeat force-pushed the knowledge/measurement-validity branch from 6ec2078 to 4458a05 Compare November 16, 2026 13:35
@BeatSeat

Copy link
Copy Markdown
Author

Thanks for the approvals. Following cursor[bot]'s merge-process note, I rebased onto current main 6c7e2379; the new head is 4458a053.

  • Only the generated files conflicted. Records were regenerated with the two-commit flow from the rebased source commit ee8c924f (regeneration f89f7667); the canonical content is unchanged from the approved head 6ec20789.
  • Installed-resource count, please note: main 6c7e2379 (feat: refine vLLM Ascend ACLGraph and model adaptation knowledge (Refs #350) #382) added one packaged knowledge resource without updating tests/check_installed.py. A wheel built from main's clean archive installs 928 files against the asserted 927, so main's installed check is already off by one. With this reference the count is 929. I set it in a separate commit, 4458a053, outside the protected paths, and verified it on a wheel built from a clean archive of this head. If main fixes its own count first, this becomes a one-line rebase.
  • build_skill_index.py --check, build_skill_coverage.py --check, knowledge_declaration_paths.py, ruff and the knowledge test set (485 passed, 5 skipped, 112 subtests) pass. An offline run of main's check_knowledge_model_policy.assess() against the updated body is compliant (declaration compliance only).

The plan check failure is the setuptools-scm tag-parse issue on main (evidence-minicpm-seedtts-dec7a734-v1), not this slice. Please re-review on 4458a053.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base 6c7e237935a22cc5e90e954d4f397376e753e7d8 and head 4458a0530a0e2a312eb6ce98af190afe5091b97e (re-review after author rebase; prior Approves on 6ec207892ed4 dismissed).
Verified #455 docs-only slice: gpu-measurement-validity.md retains unverified/source-derived status and no numeric gate; matrix rows in dedicated sections (not #60); skill pointer + regenerated pack/projections; installed count 929 matches +1 reference vs post-#382 base. Still draft and CONFLICTING vs current main after #470/#472 (check_installed 928→ needs rebase keeping 929). Mark ready only after rebase onto current main + ready CI.
Standing steward review delegation while hosted Dong-bot is on usage_limit.

@cursor

cursor Bot commented Nov 16, 2026

Copy link
Copy Markdown

Steward note: Content re-PASS on head 4458a053 after rebase (cursor Approve). Before marking ready: rebase onto current main (includes #470/#472; keep installed count 929 for the new reference vs main’s 928) to clear CONFLICTING, then ready CI.

@amy-why-3459 amy-why-3459 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base 6c7e237935a22cc5e90e954d4f397376e753e7d8 and head 4458a0530a0e2a312eb6ce98af190afe5091b97e (draft), incremental from previously reviewed 6ec207892ed4204b9e14d1fc6166681e048b301e.

Canonical content is unchanged: gpu-measurement-validity.md, the skill pointer in SKILL.md, and metadata.json are byte-identical to the previously approved head. Matrix rows remain in the dedicated "GPU measurement validity and profiler attribution" section, and docs/skill-coverage.md labels that source under that section (16 readsets; candidate / not_run preserved). Provenance source_revision is ee8c924f61f0a8f17338b8b25909a410398067ad; the installed-resource count is 929 on this head (928 actual files on base 6c7e2379 plus the new reference).

Merge process only (not a content blocker on this draft head): main has since moved to 2cbfab66 (#470, then #472, which sets the installed count to 928), and this branch is CONFLICTING. Rebase before marking ready; the fixture should then become 928→929.

I did not re-download the NVIDIA/PyTorch documents to re-check retained hashes, and I did not rerun the generators or the installed-wheel check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:knowledge Knowledge line: packs, skill library, knowledge admission kind:task Bounded delivery task with a delegated slice

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants